Weight penalties.

On Quadratic Penalties in Elastic Weight Consolidation

Elastic weight consolidation [EWC, Kirkpatrick et al., 2017] is a novel algorithm designed to safeguard against catastrophic forgetting in neural networks. EWC can be seen as an approximation to Laplace propagation [Eskin et al., 2004], and this view is consistent with the motivation given by Kirkpatrick et al. [2017]. In this note, I present an extended derivation that covers the case when the...

The Impact of Weight-based Penalties on Drug Purity and Consumption: a Theoretical Analysis

Splines, Knots, and Penalties

Penalized splines have gained much popularity as a flexible tool for smoothing and semi-parametric models. Two approaches have been advocated: 1) use a B-spline basis, equally-spaced knots and difference penalties (Eilers and Marx, 1996) and 2) use truncated power functions, knots based on quantiles of the independent variable and a ridge penalty (Ruppert, Wand and Carroll, 2003). We compare th...

Facility Leasing with Penalties

In this paper we study the facility leasing problem with penalties. We present a primaldual algorithm which is a 3-approximation, based on the algorithm by Nagarajan and Williamson for the facility leasing problem [NW13] and on the algorithm by Charikar et al. for the facility location problem with penalties [CKMN01].

Adaptive penalties for evolutionary

In this paper we consider a problem independent constraint handling mechanism, Stepwise Adaptation of Weights (SAW) and show its working on graph coloring problems. SAWing technically belongs to the penalty function based approaches and amounts to modifying the penalty function during the search. We show that it has a twofold ben-eet. First, it proves to be rather insensitive to its technical p...